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ABSTRACT 

This study investigated the effects of scorekeeping 
on student motivation in a computer assisted arithmetic drill and 
practice game. A 2x2 factorial design was used which incorporated the 
four treatments formed by time score (present and absent) and rank 
score (present and absent). Selected on the basis of their previous 
experience with the procedure of carrying in addition problems, the 
subjects were 52 second grade students who were randomly assigned to 
the four treatments. After a brief introduction and demonstration of 
the rules of the game, each student could elect to play further 
rounds to a maximum of 15 minutes. Motivation was measured by tha 
number of rounds elected. An analysis of variance calculated for the 
number of rounds played indicated that there was no significant 
treatment effect. Discussion of the results and suggestions for 
further studies conclude the report. (Author/RP) 
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ABSTRACT 

This study investigated the effects of scocekeeping on student notivation in 
a oonfjuter assisted arithnstic drill and practice game. A 2 x 2 factorial 
design was uced which incorporated the four treatments fcxined by time soDre 
(present and absent) and rank score (present and absent). Subjects were 52 
aecond grade student-^ who were randomly assigned to the four treatments. 
After a brief introduction and demonstration of the rules of the game, each 
student oould elect to play further rounds to a noKimun of fifteen minutes. 
Motivation was measured by the nurber of rounds elected. An analysis of 
variance calculated for the nunnber of rounds played indicated that there was 
no significant treatment effect. Discussion of the results and suggestions 
for further studies are provided. 
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IWTRDDOCTION 



Ihe infinite patience of the conputer and its ability to provide 
materials at appropriate levels of difficulty have contributed to its use in 
the drill and practice mode, particularly in the area of mathematics. Based 
on the premise that the student has learned the basic concept, principle or 
procedure, the computer program provides practice to enable the student to 
internalize and further develop comrand of the material. A major concern in 
the design of such programs is the aspect of motivation. While it is 
necessary to provide continuous and imnediate feedback for ccrject answers 
and constructive feedback for incorrect ansviers, it must be done in a way 
that encour^lges the student to continue at increasingly higher levels of 
difficulty and thus benefit from the program. 

In the last sevetal years, there have been increasing interest in and 
availability of drill and practice programs set in a game context. It has 
been suggested that certain aspects of games have a significant effect on 
student motivation. Attenpts have been made to identify the elements of 
computer g~aes that contribute to their ag^al and to determine their 
relative influence on motivation as well as to establish prescriptions for 
including these elements in educational games. 

To our knowledge, there have been only three studies (Malone, 1981) 
that have addressed these concerns in a controlled setting, in this ^series 
of three studies, attenpts were made to discover the features that make 
certain ccmputer games fun to play. Ihe report of these studies includes a 
theory of intrinsically motivating instruction which prescribes ways to 
inccHnporate into educational computer games the features that make computer 
games fun to play. 

Halone discovered that the two features that had tne highest 
correlations with ganne preference were the presence of a goal and scoring. 
Since the concept of a game by definition implies there be a purpose or goal 
to be achieved, it vould seem that further %*ork on goals should relate to 
the design and ccmnnunication of the particular nature of a game's goals. A 
more important area of inquiry seems to be the nature and role of 
scorekeeping in educational computer games. The feature of scoring had a 
.56 correlation {p<.01) with game preference in the studies described above. 



Scorekeeping provides several important elements to an educational 
computer game. Scoring allows players of a game the opportunity to evaluate 
tl^ir perfocmanoe by providing clear feedback that continuously lets them 
knew how well or how poorly they are doing at any given point in the game 
(Csikszentmihalyi, 1975). Students may begin play at an appropriate but low 
level of difficulty. Scorekeeping then provides the encouragement to 
proceed to increasingly difficult levels. The element of scorekeeping also 
provides an aspect of uncertainty to the game by encouraging the student to 
:^ucceed at a higher level of challenge, ihis challenge is different in 
d\fficulty but not in kind from the earlier levels of play. Because the 
::ur:?cse of drill and practice games is automatizing previously learned 
tskUls (Heinich, Molenda, and Russell, 1982), those elements that contribute 
to students* persistence on a task are essential. Since motivating feedback 
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following a response contributes to persistence and maintenance of 
perfomance (Keller, 19^3), scorekeeping appears to be an appropriate and 
intXHTtant motivational element for investigation* 

The previous three studies (Malone, 1981) have not, however, adequately 
measured the effect of scorekeeping in particular on itotivation* There were 
two najc^ problems with those studies. Ihe first is that the attenpt to 
measure the effect of several variables in a game at one :ime has, in 
effect, resulted in the comparison of essentially different games* Second, 
the studies cited have not adequately controlled for multiple-treatment 
interaction (Campbell and Stanley, 1963)* In Mdlone*s studies, a series of 
variables, which included scorekeeping, was presented to students 
consecutively and in a Simulative manner, making it impossible to obtain an 
accurate measure of the effects of each of those variables individually* 
Therefore, while claims for the influence of scorekeeping are intuitively 
appealing, the effect of scorekeeping on student motivation has not been 
properly assessed and should be considered tentative at best. 

Ihe purpose of this study, then, is to examine the effect of 
scorekeeping on student motivation in a computer drill and practice game. 
Scorekeeping is understood to be a measure of a student *s progress tcMat6 a 
goal. There are kinds of score: objective and evaluative. Foe 

exanple, a time score is objective whereas a rank score is evaluative. 
Time score is defined here as the time it takes the student to complete one 
round of the game (not necessarily real time). Rank score is defined as a 
rating the students receive which is based on their playing time. The time 
scote provides the students with objective feedback on their performance 
much like that provided someone finishing a long-distance race. Rank 
score provides a qualitative categorical rating of the students* 
performance, such as high, medium, oc low. While the time score is 
objective, reporting a measure of a performance based on a neutral scale, 
the rank score provides a connparative report of a student *s performance. In 
this study, both types of scorekeeping are manipulated while all other 
variables are held constant. 

It is hypothesized that: 

1. The presence of a time score will result in greater student 
motivation than no score at all. 

2. The presence of a rank score will result in greater student 
motivation than no score at all. 

3. The effects of time and rank score will be additive when used 
together (i.e., no interaction). 
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MEmX) 

students 



clasflS^ ;.f^l tL,^L ^^"^^ ^2 second-grade students from three 

oi!!?! suburban elementary school in central upstate New York, ihis 

sample represents above average ability and motivation with regard to math 
™. *^ average -notivation and experience with regard to the use 
as educational tools. All of the students had experience with 
cajputers in the area of math drill-and-practioe programs since first grade 
^viS*'^K?2i5S^-*^J^^^^ participating in the study. Ihe students' 
SJ^i^ included groups of two to three students working on math 

programs under the supervision of a parent volunteer. 

tte nJjSS.^ ^^^^^ .o" the basis of their previous experience with 

fS^rrf?^ ^" addition problems. While all stiKJents had 

d^^S^^mSJS^f -i-?^ carrying, because of variations in ability, seme 
nr^n^f ^T^^ ^5,""'^^^^ others. For the present study, 

students were randomly selected fron their classroans and 
teStiJg^o^!'^ to one of the four treatment groups as they entered the 

Design 

\rS^l^^'^\ experimental design and a 2 x 2 factorial design were 
usea for this study. The factors were two levels of scorekeeping-^time 

^ ^^'^^ ^"^ "'^^ <P™sent and absent). The 

st?*:istical design is shown in Figure 1. / 



Insert Figure 1 about here 

Task and Materials 



« ^T^t^ presented with an arithmetic drill and practice game 

SLSJniJ^i^ I. "^"^^ "^^^ ^^^'^ """i^*^- gate consisted of 

«^ ^^^^"^ problems which required the procedure of carrying. Each 
J^iff ^ consisted of ten addition problems. One of the ten 

ifiiSSJ o»f • ^ . top of the screen would be removed from the screen as the 
nJ^L * P»^l«"- provided as a 

^IJS- .Ti^-^'^'^'^f ^'^"^"'^ ^tep of th^ problem. If 

an error was ccwnitted in either the process of addition or carrying, an "X" 

^f^l ^ particular box to cross out the wrong answer. After a 

^^ter .^^"L.^^' "^^^ ^ erasedlnd the Student couW 

^r^^\.^^ J^J^^ response. At the end of each gane a screen menu would 
ThT^LrS^^ .^i*^ ^-^^ ^ either stop or continue play for another round. 
m^<«f ^ ""^K^^IL.^^ represented in Figure 2. The speed of the ciock was 
preset, and each problem was randomly generated by the cJinputer program. 



Insert Figure 2 about here 
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Rank Score 
prosent absent 



Time Scx>re present 



absent 



Figure 1 : 2 •< 2 Factorial design 



AAAAAAAAAA 



15 
♦56 
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Figure 2: Screen design for presentation of probleno 
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Tteatnents 

Uiere were four treatnient groups (see Figure 1 above): tune and rank 
score, time only, rank only, and no time or rank score. Itie stPicture and 
content of the game remained identical across treatment groups. At the end 
of a round the time/rank score group was presented with the tin*) it took to 
complete that round and a rank which was based on that time. The time 
score only group received only the time it took to ccrplete that rcxjnd, and 
the rank sccm:« only groip was only presented with a ranking of their 
performance. in this game, the rankings range from "slug Dug" to indicate 
a low or slow score to "super speed demon" which represents a high or fast 
score. Ihe no time/no rank group didn't receive any form of score. tho 
screen designs for the scoring of the time and rank, time only, and rank 
only treatments are provided in Figure 3» 



Insert Figure 3 about here 



Tests and Measures 

Ihe number of rounds each student played was recorded by a research 
assistant and used as a measure of motivation. Information about student 
attitude toward the game and the scarekeepirg %#ere taken in a brief 
post-treatment interview. The interview collected data regarding the 
students' satisfaction with the game, their opinion about the game's best 
feature, suggestions for making the game better, and their attitude 'toward 
the particular scorekeeping treatitent they received. The interview 
instrument is provided in Figure 4. 



Insert Figure 4 about here 



Procedure 

The students were randondy selected to report to the exper^jnent room in 
groups of four. Each student was randomly assigned to one of the four 
versions of the math drill and practice game. Pcur cottxjters were arranged 
at four areas of a large work room so that students working at one station 
would not be distracted by students working at the other stations. Before 
playing the game, research assistants located at each station presented each 
student with a brief tutorial which demonstrated the game's procedure. The 
research assistants also recorded the amcxint of playing time and the nunyaer 
of rounds played by each student. The instructions provided to each of the 
research assistants and a description of thie tutorial are provided in Figure 
5. The instructions were followed very well. 



Insert Figure 5 about here 



ERIC 



9 



r 



(SBJ Here is your time and ratings 



SUPE» SPEED DEMON 
SPEED DEMON 
-> QUICK WIT 
TFUCKER 
SUOW POKE 
SUJG BUG 



2:30 



Time/rank score 



ISBl Here is your time: 



2:30 



Time score 



I SB) Here is your rating; 

SUPER SPEED DEMON 
SPEED DEMON 
~> QUICK WIT 
IWCKER 
SL3W POKE 
SLUG BUG 



Rank score 



Figure 3: Score presentation screens for time/rank, time, 
and rank score treatments 
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08SERVATICW/INTERVIEW FORM 
GROUP STUDEKT. 



PIAYINS TIME_^ minutes MALE ( ) FE«ALE ( ) 

(15 minute maxinun) 

NUHBER OF GAMES OBSERVER 

^ASSISTANTS DO NOT WRITE BELOW THIS LINE 



1. Tfell me how veil you liked this game? Would you say you: 

1. Didn't like it at all...( ) 3. Liked it very nwch ( ) 

2. Uked it .....( ) 4. Don't knew ( ) 

2. What about this game did you like best? 



3. Can you think of anything that would make this game better (more fun 
to play)? 



4. What about this gan» made you want to keep playing? 



(TR) 5a. Did seeing your time score (how long it took you to play each gatw) 
make you want to keep playing the garoe? 

Yes ( ) No ( ) Dc3n't know ( ) 

(TR) 5b. Did seeing your rating (slug bug, speed demon, ei;c.) make you want 
to keep playing the game? 

Yes ( ) No ( ) Don't know ( ) 

(T) 6. Did seeing your time score (how long it took you to play each game) 
make you want to keep playing the game? 

Yes ( ) No ( ) Don't know ( ) 

(R) 7. Did seeing your rating (slug bug, speed demon, etc.) make you want 
to keep playing the game? 

Yes ( ) No ( ) Don't know { ) 

(0) 8a. If you had' seen your time score (how long it took you to play each 
game), would it have made you want to keep playing? 

Yes ( ) No M Don't know ( ) 

(0) 8b. If you were given a rating (like slug bug, speed demon, etc.), 
would it have made you want to keep playing? 

Yes ( ) No ( ) Don't know ( ) 



O Figure 4: Observation/Interview Instrument 
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PRXESURE: 

««>You : ASSISTANT) have four things to do: 

1. Say hello and introduce yourself (first names) to 

the student as they sit down in front of the cotputer. 

You Tflay tell them that this is a new math gane and 

we are trying it out to see how much fun it is to play. 

2. Present the rules of the game to the students. ("Here's 
how it works.") 

3« On your observation sheet , record the length of time 
they play the game (to a maximum of 15 minutes) AND 
the nunnber of games they play. 

4. When they are done playing the gamei congratulate the 
student for a good job and take them out to the hall 
(with your observation form) to talk to Mr. %>uches. 

As your first student is seated at your ccn|)uter, the computer progrew will 
have already been booted and the following screen will be displayed: 

(Would you like me to tell you the rules?] 

~> Assistant types "Y" 

As the program explains the rules of the game you should read the rules out 
loud to the student. The rules are relatively siiqple and most of the 
students will have had some experience with this type of game so you 
shouldn't need to spend very long on this. Be attentive to the student and 
proceed to the next frame after you have confirmed tiiey are with you (a 
sinple OK? should do it). 

This deioo should take approximately 60 seconds. 

fr will show you a problem. You must type the answer 
as quickly as you can -SB-] 

~>Assistant types SPACE EAR 

(First add the ones -SB-) 
««>Assistant types SPACE BAR 

(Next we carry the ten by typing a "l** -SB-1 
««>Assistant types SPACE BAR 



Figure 5: Instructions to Rjsearch Assistants 
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(Ihen we edd the tens and type in the answer 
from right to Teft) 

At this point paust briefly to let the student know you are ready to 
begin ("Ready to start?" should do it), 

»>A£sistant types SPACE BAR 

{I will keep track of your ♦ otal tine. 
GET READY (repeated THREE times] 

•*aw5>Begin timing the stuc^t as soon as the first problem appears on the 
screen. The student will now play the game. 

AND 

**«»'>Recotd the nunnber of games they p^ ly. 

At the end of eadi game the student may be presented with their time and/or 
rank score or no score at allr depending on the version they are playing. 
If a score is presented follow the next directions. (If no score is 
presented you will skip the following step and be taken to the student 
choice screen). 

After their score is displayeO^ pause for about tixee seconds # 
say "OK" and 
««>type SPACE BAR 



(You may choose what you would like to do next. 

1. Play speed game again 

2. Stop playing] 

The student should type in their choice. If they do not respond to the 
screen or seem confused^ sinply read tlie sc ^en to them and remind them to 
type in their choice. 

»>CQmplete your observation form. Congratulate the studen. for a good jcto 
and bring them to Mr. Spuches along with you oteervation form. 

«»>If you haven •t already it, press No. 2 ("STOP PLAYING*) and 
SPACE BAR. This will bring ^ program to the explanation of the 
gamers rules for the next stuJer^t. 



While you dxxild be pleasant and relaxed with the students, your interaction 
and dialogue shcxild closely follow the outline abova. Don*t congratulate or 
encourage students until they have completely finished. 



Figure 5; continued 
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Poliowing the-se instructions, each student was allowed to play as many 
rounds as they wished, up to a maximum of 15 minutes. After each round, a 
screen di^layed the qption for the student to either play another round or 
to stop playing* 

Mhen a student indicated that U^ey wanted to stop playing, the number 
of rounds played was recorded, and the student was brought to a near-ty 
location and interviewed individually by the researcher^ 

RESULTS 

Measures of Motivation 

The average number of rounds played in each of the four treatment 
groif)s is ohown in Table 1. It had been hypothesized that the richer 
treatment, i«e«, the combination of time and rank score, would result in 
increased motivation for the students to play the math drill and practice 
game* However, the analysis of variances revealed that all differences were 
far fron significant* 



Insert Table 1 about here 



Measures of student attitude recorded in the post-treatment interview 
%iere consistent with the means for the nurber of games played* As shown in 
l^le 2, students in the combined time/rank score treatment indicated the 
poorest overall attitude toward the game* In contrast, students in the 
no-score trr iitment appeared to derive the most enjoyment directly from 
playing the math gam* This group indicated strongly that they found the 
opportunity to use and review previously learned maith skills in a game 
fonnat to be the most attractive feature* In response to the question 
regarding the particular aspect of the game that made them want to keep 
playing, the no-score treatment, again by a wide margin, indicated their 
direct enjoyment of working with math skills and problems* Mhile a 
considerable percentage of the students in both the time and the rank score 
treatment groups indicated they found that the aspect of trying to get a 
better score in succeeding games made them want to keep playing, few 
students in the time/rank treatment group said this was an inportant aspect* 



Insert T^ble 2 about here 



DISCUSSION 

Math drill and practice games appear to be the most frequently used 
fonn of computer assisted instruction* There is much interest in the 
plication of ccmputer game features to make these and all educational 
conputer gsr«as more enj^^able as well as educational* Previous studies have 
claimed that the feature of soorekeeping is highly influential on student 
motivation in such contexts* They have not , however, controlled for other 
variables while measuring the effect of scoring on motivation* In the 
present study, soorekeeping was operationalized as a time score and a rank 
score and, based on previous research ard theory, it ms hypothesized that 
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liable 1: Means, (SD), and n for the nurtber of rounds played. 



Tims 



No 



Yes 



No 



Rank 



Yes 



3.08 


3.<0 


(.50) 


(.55) 


13 


13 


3.3\ 


2.80 


(.43) 


(.50) 


13 


13 


3.20 


3.08 


(.32) 


(.38) 


26 


26 



3.23 

(.36) 

26 



3.04 
(.34) 

26 



F = .15 
p « .70 



F » .05 

P " '82 Interaction: 

F - .70 
p « .41 • 
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liable 2 : Sunmary of selected responses f ran the post-treatment interview 
(in peroents). 

1. TBll me hc3w well you liked this game. Vtoulil you say you: 





Time 


Rank 


Time/Rank 


No Score 


A. Didn't like it at all 


0 


0 


0 


0 


B. Liked it 


30.77 


23.07 


46.15 


23.07 


C. Liked it very much 


69.23 


76.92 


53.85 


76.92 


D. Don't know 


0 


0 


0 


0 


2. What about this garoe did you like best? 








Time 


Rank 


Time/Rank 


No Score 


Enjoyed using/reviewing 
math skills and playing 
math games 


61.53 


69.23 


61.53 


84.61 


Challenge; Liked trj-ing 
to figure the problems 
out real fast 


7.69 


0 


15.38 


15.38 


4. Wiat a^out this garoe made you vant to keep playing? 






Time 


Rank 


Time/Rank 


No Score 


f\jn to play; 
Knew addition and 
carrying well; 
Fun to practice 


30.76 


46.15 


30.77 


69.21 


lb see if I could 
get a better score 


38.46 


23.07 


15.38 


7.69 


Misc. 


0 


15.38 


0 


7.69 


Don't know 


7.69 


7.69 


15.38 


0 


NA~cne gams only 


23.07 


7.69 


38.46 


15.38 
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the presence o£ either tin© score or rank score would result in greater 
student motivation than no score at all# and that the oont>ined use of time 
and rank score wcxild provide greater motivation than no score at all. 

The first and second hypotheses, that the presence of either time score 
or rank score will result in greater student motivation than no score at 
all# *^re not confinned. Noreover, although a high percentage of students 
in the rank score treatment gtoqp indicated a positive attitude toward tte 
^ame# an equal percentage of students in the no<-score treatment indicated 
that they, too, had a positive attitude toward it. Xie percentage of 
students in the time score treatment %4io indicated a high level of 
satisfaction with the game was lower than that tor the rank and no-«x>re 
treatments. 

Ihe third hypothesis^ that the effects of time and rank score will be 
additive when used together, was not confirmed. Since a level of 
significance was not readied, the results suggest that the expectation for 
an additive effect to occur when time and rank score are used together may 
be unreasonable, at least under the present conditions. 

The results of this study suggest the following conclusions. 
Elementary school students find the actual use of math skills and their 
practice in a gatne^like situation to be intrinsically motivating and the 
introduction of scoring may actually inhibit the intrinsic enjoymmt derived 
from this activity. Also, the use of ocnfHiter assisted drill and practice 
games should be used to enhmce students* camiand of the basic skills and 
may be, in and of themselves, highly motivating. The use of soc»pek^ing, 
however, requires further investigation, especially with regard to its 
relation to student characteristics. Future studies, then, should not 
address the question '"is the use of sccxrekeeping an appropriate motivational 
element in computer assisted educational games,** butjrather "when and for 
whcxn are the various types of scorekeeping useful?** 
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